“How  Much  is  a  Pound  of  C4ISR  Worth?” 


An  Assessment  Methodology  to  Evolve 
Network  Centric  Measures  and  Metrics  for  Application  to  FORCEnet 


Mr.  John  A.  Poirier 

Director,  Program  Integration  and  Outreach 
Senior  Policy  Analyst 

Scientific  Applications  International  Corporation 
4001  N.  Fairfax  Drive,  Suite  500 
Arlington,  VA  22203 

Mr.  Edgar  Bates 

Senior  Analyst 

Scientific  Applications  International  Corporation 
Capability  Assessments 

Space  Information  Warfare  Command  and  Control  (N61) 
Chief  of  Naval  Operations 
2000  Navy  Pentagon 
Washington,  DC  20350-2000 

CAPT  Mark  Tempestilli,  USN 

Director  of  Network  Centric  Warfare 
United  States  Naval  Academy 
290  Buchanan  Rd. 

Annapolis,  MD  21402 


1 


Report  Documentation  Page 

Form  Approved 

OMB  No.  0704-0188 

Public  reporting  burden  for  the  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources,  gathering  and 
maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection  of  information, 
including  suggestions  for  reducing  this  burden,  to  Washington  Headquarters  Services,  Directorate  for  Information  Operations  and  Reports,  1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington 

VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  a  penalty  for  failing  to  comply  with  a  collection  of  information  if  it 
does  not  display  a  currently  valid  OMB  control  number. 

1.  REPORT  DATE 

JUN  2003  2' REPORT  TYPE 

3.  DATES  COVERED 

00-00-2003  to  00-00-2003 

4.  TITLE  AND  SUBTITLE 

’How  Much  is  a  Pound  of  C4ISR  Worth?’  An  Assessment  methodology  to 
Evolve  Network  Centric  Measures  and  Metrics  for  Application  to 
FORCEnet 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHOR(S) 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS (ES) 

Scientific  Applications  International  Corporation, 4001  N.  Fairfax  Drive 
Suite  500, Arlington, VA, 22203 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS  (ES) 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 

Approved  for  public  release;  distribution  unlimited 

13.  SUPPLEMENTARY  NOTES 

The  original  document  contains  color  images. 

14.  ABSTRACT 

15.  SUBJECT  TERMS 

16.  SECURITY  CLASSIFICATION  OF:  17.  LIMITATION  OF 

\PlSTR 

1 8 .  NUMBER  1 9a.  NAME  OF 

op  PAHRQ  PR9Pn\T9Tm  p  prpchm 

a.  REPORT  b.  ABSTRACT  c.  THIS  PAGE 

unclassified  unclassified  unclassified 

51 

Standard  Form  298  (Rev.  8-98) 

Prescribed  by  ANSI  Std  Z39-18 


ABSTRACT 


This  paper  offers  a  foundation  for  an  assessment  framework  involving  C4ISR  processes 
in  FORCEnet.  It  collects  and  expands  upon  work  done  by  a  number  of  organizations 
engaged  in  efforts  to  structure  a  process  that  links  traditional  and  evolved  C4ISR 
attributes,  measures,  and  metrics  to  network  centric  outcomes  including  work  being  done 
in  support  of  the  Office  of  Force  Transformation  and  the  office  of  the  Assistant  Secretary 
of  Defense  for  Networks,  and  Information  Integration  (ASD  (NI2)).  Using  a  capabilities 
based  approach  to  analysis,  metrics  are  recommended  and  developed  to  measure 
organizational  performance.  Performance  improvements  such  as  increased 
responsiveness  and  efficiency  are  measured  in  the  context  of  Situation  Awareness  (SA), 
in  a  form  consistent  with  the  framework  of  a  Capability  Maturity  Model.  This  paper 
offers  five  levels  of  Shared  Situational  Awareness  that  can  provide  the  basis  for  assessing 
an  organization  in  a  specific  domain  relevant  to  FORCEnet.  These  resources  are  intended 
to  guide  an  organization  in  implementing  a  series  of  increasingly  sophisticated  practices 
and  activities  that  can  have  a  significant  impact  on  individual,  team,  unit,  and 
organizational  performance 


Introduction 


Network  Centric  Warfare  (NCW)  is  real,  and  it  is  daunting — not  because  it  is  new,  but 
because  of  the  apparent  complexity  it  introduces  both  in  conduct  of  operations  and  the 
means  to  measure  the  value  of  contributing  elements.  It  removes  many  of  the  physical 
and  geographic  boundaries  that  have  historically  driven  military  operations  and  decision 
making.  It  draws  upon  resources  including  people,  platforms,  systems,  and  organizational 
processes  in  ways  that  provide  unprecedented  flexibility  and  agility  to  warfighters.  It 
combines  these  resources  to  provide  tailored  “packages”  of  capabilities  to  meet  transient 
operational  requirements  in  complex,  dynamic  environments.  It  links  human, 
technological,  and  organizational  “systems”  in  an  intertwined  and  co-evolutionary 
process  that  crosses  physical,  information,  cognitive,  and  social  domains.1  Moreover,  it 
does  so  in  ways  that  often  are  responsive  to  novel  and/or  unanticipated  circumstances. 
And,  it  is  hard  for  many  to  describe  NCW  succinctly  as  a  concept  that  can  be 
“ operationalized 

While  jargon,  this  term  reflects  the  fundamental  endgame — the  ability  to  take  this  new 
concept  and  implement  it  as: 

•  A  set  of  business  processes, 

•  Warrior  development  methodologies  (e.g.,  recruitment,  training  and  education, 
professional  development,  retention), 

•  A  changed  philosophical  or  doctrinal  view  of  operations, 

•  Expansion  of  access  to  tailored  resources  by  individuals  at  all  operational 
echelons,  and 


1  See  Alberts,  David  S.,  Information  Age  Transformation:  Getting  to  a  21st  Century  Military,  Washington, 
DC:  CCRP.  2002 
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•  Systems,  technologies,  and  organizational  infrastructure  that  support  best  the 
cognitive  and  decision  requirements  of  a  globally  distributed  human  system. 

This  is  a  major  challenge  facing  the  US  Navy  as  it  seeks  to  implement  FORCEnet,  the 
instantiation  of  NCW  in  a  naval  (US  Navy  and  US  Marine  Corps)  context.  FORCEnet  is 
central  to  achieving  the  transformational  operational  processes  articulated  in  Sea  Power 
21  comprising  Sea  Strike  (projection  of  offensive  military  capabilities),  Sea  Shield 
(projection  of  defensive  capabilities),  and  Sea  Basing  (forward  presence  of  military 
capabilities  and  resources  independent  of  national  boundaries).  Sea  Power  21  enables 
these  processes  through  Sea  Trial  (concept  development,  experimentation,  and 
validation),  Sea  Enterprise  (business  processes  that  empower  and  equip  the  warfighters 
with  needed  capabilities),  and  Sea  Warrior  (development  of  a  force  with  the  knowledge, 
skills,  and  abilities  to  function  effectively  across  the  range  of  military  operations). 

FORCEnet  binds  all  of  the  Sea  Power  21  elements  in  a  melded  human-technology  system 
of  systems.  It  is  this  system  of  systems  view  that  presents  a  number  of  challenges  in 
assessing  operational  value.  Traditional  concepts  regarding  attrition-based  measures  must 
be  thought  through  to  identify  means  to  assess  operational  outcome  in  a  way  that  yields  a 
valid  comparison  between  platform-centric  and  network  centric  capabilities.  And  here, 
the  conceptual  difference  is  the  location  of  the  seam:  at  the  “lifelines”  in  platform-centric 
operations,  across  function  in  NCW  and  FORCEnet.  Interoperability  becomes  a  legacy 
concept  and  integration  the  core  design  principle,  a  principle  requiring  measurement  of 
value,  alignment,  and  tradeoff  decisions  regarding  a  host  of  capabilities. 

Military  command,  control,  communication,  computers,  intelligence,  surveillance,  and 
reconnaissance  (C4ISR)  architectures  particularly  must  respond  to  a  number  of 
“reconstructed”  operational  challenges.  Many  have  historical  underpinnings.  Others 
require  a  comprehensive  rethinking  of  operational  and  organizational  dynamics  to 
achieve  Sea  Power  21.  Specifically  challenging  are: 

•  Asymmetrical  force  structures 

•  Transforming  attrition  based  “targeting”  into  effects  based  concepts, 

•  Asynchronous  interactions  worldwide  in  a  netted  environment, 

•  Response  to  globally  distributed  threats  requiring  coordinated  and  mutually  aware 
globally  distributed  resources, 

•  New  ways  of  using  “old”  platforms  and  capabilities  and  means  to  compare  effect, 

•  Leveraging  interagency  resources  and  information  exchange,  and 

•  The  criticality  of  properly  empowered  decision  making  at  all  organizational 
levels. 

This  paper  provides  an  approach  to  address  these  challenges.  It  consists  of  an 
introduction,  description  of  the  challenge,  a  notional  basis  for  an  analytical  framework, 
means  for  performance  based  measurement  using  a  maturity  model  assessment  approach, 
and  suggested  applications  throughout.  The  approach  includes  use  of  taxonomic  and 
ontological  constructs  in  order  to  improve  FORCEnet  C4ISR  systems  acquisition, 
decision  support,  course  of  action  analysis,  and  other  tool  development. 
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The  Challenge 


Central  to  defining  a  toolset  and  to  ensuring  the  availability  of  appropriate  capabilities 
packages  in  the  operational  space,  is  the  need  for  heuristics  based  upon  valid,  reliable, 
and  credible  measures  and  metrics  that  help  answer  the  essential  question  of  "what  is  a 
pound  of  C4ISR  worth?" 

Answering  this  question  is  a  significant  challenge  as  the  value  of  C4ISR  systems  is  in  the 
“eye”  of  the  individual  assessing  them.  The  challenge  is  compounded  by  novel  conditions 
in  network  centric  operations  (NCO)  and  effects  based  operations  (EBO)  that: 

•  Blur  traditional  operational  and  organizational  lines  of  communication  and 
authority; 

•  Emphasize  rapidity  of  information  processing,  decision  making,  and  operational 
tempo;  and 

•  Depend  fundamentally  upon  the  ability  to  develop  and  manage  knowledge  to 
achieve  discrete  outcomes. 

For  purposes  of  this  paper  measures  provide  the  vehicles  by  which  things  might  be 
assessed  and  metrics  the  mathematical  dimensions,  capacity,  and  amount  of  some  thing, 
process,  or  effect.  A  measure  in  this  case  might  be  latency  from  information  availability 
to  decision  made  in  a  “decision  cycle.”  A  related  set  of  metrics  might  include  time  in 
specific  units,  number  of  nodes  through  which  information  may  have  to  pass,  and 
effectiveness  of  engagement  (was  the  threat  neutralized?).  Others  may  include  decision 
velocity/volatility,  information  velocity/volatility,  and  knowledge  velocity/volatility. 
Here,  velocity  in  terms  of  both  how  quickly  information  was  passed  to  a  specific  node  of 
interest  and  volatility  in  terms  of  how  many  times  the  information  changed  its  content 
and  relevance  are  addressed.  It  may  be  used  as  a  point  of  comparison  between  how 
today’s  battlegroup  and  marine  force  achieves  a  desired  mission  (e.g.,  air  defense)  and 
how  a  future  “capabilities  bundle”  uses  resources  to  achieve  a  desired  effect  (no  air 
threat).  These  examples  provide  an  indication  of  a  bridge  between  traditional  and  future 
approaches. 

A  number  of  general  C2  design  considerations  also  emerge.  These  include  preparation 
and  training;  scope  of  the  battlespace;  shared  awareness;  integration;  decision  support; 
decision-making;  self-synchronization;  dispersal;  agility;  simultaneity  of  processes;  and 
continuous  and  simultaneous  operations.  Together,  these  result  in  a  need  for  new  metrics 
and  new  ways  to  apply  existing  metrics  to  determine  effectiveness  and  performance  in 
dynamic  complex  environments.2 

While  it  is  impossible,  and  unnecessary,  completely  to  decompose  the  operational  space 
into  its  basic  structure,  it  is  necessary  to  provide  a  framework  responsive  to  challenges 


2  From  C2  Literature  review,  page??  outlined  in  the  Joint  Staff  J3/Joint  C4ISR 
Decision  Support  Center’s  Joint  Task  Force  Command  and  Control  Operational 
Concept  Study  Phase  I  Final  Report 


4 


affecting  the  degree  of  achievement  of  the  needed  capabilities.  We  suggest  several  initial 
analytical  dimensions  that  may  be  developed  as  an  ontological  basis  to  guide  this 
assessment. 

First  is  analysis  based  on  perspective  (individual,  organizational,  and  systems).  Second  is 
analysis  based  on  knowledge  management  processes  (cognitive,  collaborative,  and 
integrative).  Third  is  analysis  based  on  operational  scope  (tactical,  operational,  and 
strategic).  Fourth,  is  analysis  based  on  network  centric  “domains”  (cognitive, 
information,  and  physical).  Synthesis  across  these  structures  results  in  aggregated 
assessment  of  capabilities  to  sense,  understand,  influence,  act,  visualize,  and  effect  that 
are  central  to  C4ISR  value  assessment  in  future,  as  well  as  current,  military  operations. 
They  also  provide  a  self-checking  function  to  ensure  relevance  of  the  metrics  to  a  specific 
situation. 

To  expand  on  this  notion,  enduring  elements  critical  to  enabling  naval  operations  provide 
a  point  of  departure  for  developing  a  C4ISR  taxonomy.  These  are: 

•  Ability  to  sense  through  globally  distributed  technological  and  cognitive  systems; 

•  Ability  to  understand  through  individual  perception,  shared  awareness,  and 
aggregation  of  these  elements  in  organizations  to  result  in  knowledge 
mobilization; 

•  Ability  to  influence  friend,  foe,  and  others  by  conditioning  of  the  operational  space 
through  behaviors,  lines  of  authority,  positioning,  and  posture; 

•  Ability  to  act  rapidly,  decisively,  and  appropriately  as  coordinated  across  tactical, 
operational,  and  strategic  echelons; 

•  Ability  to  visualize  the  operational  space  in  ways  that  are  relevant  to  individual 
users;  and 

•  Ability  to  effect  desired  outcomes. 

Each  offers  taxonomic  offshoots  to  collect  existing  metrics,  suggest  new  ones,  and  guide 
the  user  to  determine  appropriate  measures  to  be  used  in  a  given  situation.  At  the  end  of 
the  day,  however,  the  practical  requirement  is  for  straightforward,  flexible,  and  relevant 
tools  that  help  operators  determine  what,  where,  when,  and  how  to  access  C4ISR 
resources  as  they  are  needed.  These  tools  must  be  sufficiently  intuitive  that  they  allow  for 
ready  translation  in  the  minds  of  operators  to  tangible  application  in  the  environments 
they  face.  They  must  retain  familiar  elements  that  allow  individuals  to  leverage  their  base 
in  experience,  while  fostering  their  ability  to  innovate  in  the  face  of  new  challenges. 
These  operator-focused  tools  in  turn  must  drive  development  of  related  tools  that 
properly  inform  service  and  DOD  level  program  and  acquisition  decisions.  Here,  scarce 
procurement  resources  necessitate  trades  based  on  perceived  value  in  process, 
technology,  organization,  and  individual  human/warrior  domains — value  that  must  be 
measured. 

Given  this,  the  abilities  to  understand,  influence,  and  act  are  the  primary  focus  of 
discussion.  They  particularly  provide  an  overarching  view  of  the  network  centric  space 
driving  all  C4ISR  processes  as  well  as  assessment  of  their  value.  As  increasing  specificity 
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is  required,  layers  in  a  taxonomy  can  be  derived  from  these  elements  that  are  relevant  to 
analytical  requirements  that  help  determine  value  in,  and  across,  contexts.  Within  this 
space,  attributes  can  be  determined  and  grouped  based  on  a  set  of  mapping 
considerations. 

The  first  element,  understanding,  is  a  knowledge  management  challenge  based  upon 
critical  processes  including  access  to  relevant  information,  transforming  that  information 
into  a  personal  context  applicable  to  the  broader  situation,  and  determining  links  to 
operational  objectives,  or  effects,  that  must  be  accomplished  in  a  specific  time  and 
space — ultimately  relying  upon  SA  and  the  ability  to  share  it. 

Compounding  the  challenge  is  the  need  for  military  C4ISR  systems  to  provide 
information  flow  to  support  understanding  at  several  interdependent,  but  very  different 
levels:  individual,  organizational/group,  and  systems  (comprising  technological  and 
human  elements).  Mapping  these  perspectives  to  specific  processes  such  as  cognition  and 
learning,  collaboration,  and  integration  is  an  important  step  towards  establishing  a  needed 
ontology.  In  terms  of  generating  understanding,  C4ISR  systems  provide  value  in 
establishing  relationships  between  perspectives  and  processes  that  are  useful  as  a 
foundation  for  metrics  grouping.  These  relationships  can  be  articulated  as  (a)  Individual- 
Cognitive,  (b)  Group-Collaborative,  and  (c)  Systems-Integrative. 

Attributes  associated  with  understanding  in  this  construct  then  can  be  defined,  and 
metrics  determined  that  provide  relevance  across  changing  contexts.  It  should  be 
emphasized  that  the  last  provides  for  the  function  of  artificially  intelligent  systems  as 
well. 

The  second  element,  influence,  is  a  prioritization/utility-based  analysis  determined  by 
location  in  the  “network.”  It  requires  consideration  of  node,  or  unit  based,  interests  (how 
important  is  accomplishing  x  to  my  or  others’  survival,  mission  accomplishment, 
achieving  an  effect?).  It  also  requires  broader  consideration  of  joint/coalition  force 
requirements  in  a  specific  operational  circumstance  using  metrics  that  foster  a  Blue-Red- 
Green  comparative  framework.  This  ability  should  provide  translation  to  assessment 
criteria,  course  of  action  analysis,  and  decision  support  tools  and  model  development.  It 
should  result  in  identification  of  both  physically  proximate  and  virtual  resources  as  well 
as  the  means  for  applying  them  to  exert  desired  influence  and  minimize  the  ability  of 
others  to  exert  negative  influence  on  oneself.  Finally,  this  element  should  be  derived  from 
an  effects  based  calculus  that  considers  political,  economic,  cultural,  social, 
infrastructural  as  well  as  military  dimensions  including  second  and  third  order 
propagation  of  effects.  It  should  not  lose  sight  of  the  fact  that  in  many  cases  attrition 
based  concepts  still  constitute  effects.  Consideration  of  system  performance  in  terms  of 
assessed  reliability,  reachback,  processing  capability,  bandwidth  requirements,  and  many 
other  areas  are  still  relevant. 

The  third,  action,  is  a  resources/capabilities  assessment  tied  to  a  window  of  opportunity. 
What  do  I  need  to  do  what  I  want  to  do  and  where  is  it?  Metrics  of  access,  readiness,  and 
distribution  are  relevant  here  as  well  as  means  to  "mass"  resources/capabilities  to  achieve 


6 


desired  effects  including  application  of  “heat,  blast,  and  fragmentation”  using  kinetic 
weapons.  This  ability  should  also  reflect  options  generated  for  "how"  to  act.  When  is  a 
kinetic  option  in  a  purely  military  context  correct?  What  alternatives  may  be  preferable  to 
traditional  tactics,  techniques,  and  procedures?  The  latter  raises  questions  associated  with 
self-synchronization  as  well  as  formal  and  informal  organizational  function  in  a 
distributed  collaborative  environment. 

Together  these  elements  provide  both  a  taxonomic  and  ontological  core  for  a  model  or 
toolbox  of  evolved  C4ISR  metrics.  Moreover,  a  number  of  “tools” — measures  and 
metrics — are  already  available  to  be  collected.  Numerous  service,  joint,  DOD,  and 
industry  organizations  have  worked  together,  but  more  often  independently,  to  develop 
attributes,  measures,  and  metrics  relevant  to  C4ISR.  These  reflect  a  range  of  perspectives 
that  comprise  techno-centric,  human-centric,  and  systems  integration  foci. 

An  issue  must  be  addressed,  however.  Often  similar  metrics  are  applied  in  radically 
different  ways  resulting  in  diverse  or  conflicting  analytical  conclusions.  Many  are 
predicated  on  traditional  concepts  and  structures  although  some  have  been  developed  to 
support  emerging  NCO  and  EBO  theory.  This  must  be  resolved  in  a  post- 9/11 
environment  with  time-compressed  windows  of  opportunity  (and  vulnerability).  That 
moment  in  time  became  a  catalyst  to  realizing  that  fundamentally  different  drivers 
characterize  the  current  operational  environment  and  that  NCW  is  more  than  a  theory. 
What  is  needed  now  is  an  assessment  methodology  that  provides  a  consistent  and 
common  vehicle  with  which  to  determine  the  value  of  C4ISR  capabilities  in  a  changed 
and  continuously  changing  world. 

Framework  for  Assessment 


Thus  far,  use  of  the  term  “capability”  has  been  without  precise  definition.  However, 
before  any  capability  measurement  in  FORCEnet  can  be  addressed,  it  is  essential  to 
derive  a  common  definition  and  concept  of  a  capability  at  least  as  it  relates  to  FORCEnet. 

For  our  purposes,  we  define  a  capability  as  that  combination  of  human,  technological, 
organizational,  process,  and  cognitive  elements  that  provides  the  means  to  achieve  a 
clearly  articulated  outcome  in  a  defined  context.  This  has  significant  implications  in  that 
it  requires  the  ability  to  link,  delink,  and  relink  these  elements  in  ways  that  are  tailored  to 
unique  circumstances,  often  in  what  are  called  “mission  capabilities  packages  (MCPs).” 

It  further  embeds  the  human  individually  and  as  part  of  an  organization  in  a  complex 
system  of  systems.  It  also  requires  that  intangible  cognitive  processes  be  accepted  and 
leveraged  as  part  of  the  system.  These  latter  include  elements  of  emotion,  “gut  feel,”  and 
other  humanistic  dimensions  that  fall  more  under  the  “art”  than  the  “science”  of  military 
operations.  It  also  should  be  noted  that  a  capabilities  based  approach  is  a  fundamental 
transformation  in  defense  planning.  The  emphasis  is  no  longer  on  the  “who”  (i.e. 
traditional  threat  based  approach)  but  rather  on  the  “how”. 3 


testimony  of  Art  Cebrowski,  Director,  Office  of  Force  Transformation, 
before  the  Senate  Armed  Services  Committee’s  Emerging  Threats  and 
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In  the  case  of  FORCEnet,  a  hierarchical  capability  taxonomy  has  been  developed  and 
refined  over  the  past  year  to  articulate  aspects  of  what  comprises  “FORCEnet.”  This 
hierarchy  consists  of  six  capabilities  at  the  top  level  as  indicated  in  Table  1.  To  facilitate 
analyses  related  to  these  capabilities,  elements  of  the  Chief  of  Naval  Operations  staff 
responsible  for  implementing  FORCEnet  have  developed  an  initial  analytical  framework 
that  also  is  consistent  with  Assistant  Secretary  of  Defense  for  Networks,  and  Information 
Integration  (ASD  (NI2))  NCW  concepts  and  analytical  resources.  This  framework  further 
couples  newer  concepts  with  existing  metrics  and  systems  performance  assessment 
criteria  associated  with  the  Universal  Joint  Task  List  (UJTL)  and  service  based  Mission 
Essential  Task  Lists  (METLs).  A  number  of  traditional  measures  and  metrics  also  may  be 
applied  to  analysis  of  the  FORCEnet  core  capabilities.  Table  1  includes  descriptions  of 
the  six  FORCEnet  core  capabilities  and  identifies  “assessment  criteria”  that  reflect  the 
mapping  of  C4ISR  operational  attributes  to  notional  metrics.  These  metrics  have  been 
drawn  from  an  initial  review  of  several  C4ISR  research  efforts  which  include  a  recent  C2 
Concepts  and  Experimentation  Literature  Review  sponsored  by  the  Joint  C4ISR  Decision 
Support  Center,  the  Joint  C4ISR  Battle  Center’s  Assessment  Methodology,  the  ASD/C3I 
Architecture  Working  Group,  the  National  Security  Agency/Defense  Information 
Systems  Agency  sponsored  Information  Assurance  Technical  Framework,  and  Defense 
Planning  Guidance.  In  general,  the  metrics  are  evolving,  and  it  should  be  recognized  that 
in  some  cases  an  attribute  could  be  further  operationalized  in  order  to  develop  a 
meaningful  metric.  Many  of  the  metrics  and  measures  mentioned  earlier  in  the  paper 
provide  additional  candidates  for  inclusion  in  the  evolving  framework  as  well. 


1.  Provide  expeditionary,  multi-tiered  sensor  and  weapon  information:  The 

expeditionary,  multi-tiered  sensor  and  weapons  grid  capability  uses  a  full  spectrum  of 
manned  and  unmanned  vehicles,  platforms,  sensors  and  weapons  to  provide  the  Force 
Commander  with  what  is  needed  to  locate  targets  and  attack  them  across  the  depth  and 
breadth  of  a  theater-sized  battlespace.  Sensors  must  determine  their  position,  time  and 
movement  at  the  precise  time  they  are  reporting  their  target  or  other  intelligence 
information.  The  time  and  position  information  of  the  track  provided  by  sensors  in  the 
grid  must  be  properly  attributed  (e.g.,  linked  to  a  standard  reference  frame  with 
uncertainty  (error)  and  confidence  level)  for  it  to  be  accurately  understood,  represented 
and  fused  with  other  data  /  information.  Many  modem  weapons  are  also  dependent  on 
precise  time  and  position  (including  uncertainty)  for  effective  operation. _ 

Attribute _ Notional  Metric _ 

Accuracy  Correspondence  with  ground  truth-correlation  coefficient  (0=  no 
correspondence  with  ground  truth,  1=  full  correspondence  with  ground 
truth).  Data  matrix  comprised  of  relevant  information  items  estimates 
(for  instance:  detection,  ID,  velocity,  location,  heading,  etc.) 


Capabilities  Subcommittee,  14  March  2003 

http://www.oft.osd.mil/librarv/librarv  files/trends/trends  30  Transformation%20Trends- 

- 1 7%20March%20Issue.pdf 


8 


Consistency 

Degree  of  lack  of  ambiguity  with  previous  information 

Completeness 

Percentage  of  ground  truth  relevant  and  necessary  for  ongoing  task 

Precision 

Error  and  confidence  level  for  time  and  position  information  compared 
to  a  standard  reference 

Timeliness 

Degree  to  which  currency  matches  what  is  needed  (0=no  match,  l=high 
degree  of  matching  between  currency  level  needed  and  available) 

2.  Conduct  distributed,  collaborative  Command  &  Control:  To  collaborativelv 

manage  land,  air,  sea,  and  space  operational  forces  in  time,  space,  and  purpose  to 
produce  maximum  relative  combat  power  and  minimize  risk  to  own  forces.  This 
activity  ensures  all  elements  of  the  operational  force,  including  supported  agencies’  and 
nations’  forces,  are  efficiently  and  safely  employed  to  maximize  their  combined  effects 
beyond  the  sum  of  their  individual  capabilities. 

Attribute 

Notional  Metric 

Shared 

Situational 

Awareness 

Degree  to  which  the  different  individual  mental  models  of  the 
situation  are  integrated  into  a  common  operational  picture. 

Quantity  of 

Posted 

Information 

Percent  of  collected  information  posted 

Quantity  of 
Retrievable 
Information 

Percentage  of  nodes  that  can  retrieve  various  sets  of  information. 

Understandability 

Degree  to  which  information  is  easy  to  use  (0=low  degree  of  ease  of 
use,  l=high  degree  of  ease  of  use) 

Precision 

Error  and  confidence  level  for  time  and  position  information 
compared  to  a  standard  reference 

Timeliness 

Degree  (speed  of  effect)  to  which  currency  matches  what  is  needed 
(0=no  match,  l=high  degree  of  matching  between  currency  level 
needed  and  available) 

3.  Provide  dynamic,  multi-path  and  survivable  networks:  To  provide  data  and 

information  flow  seamlessly  and  transparently  to  the  warfighter  across  a  fault  tolerant, 
adaptable,  self-organizing,  holistically  engineered  continuously  available  network.  The 
data  and  information  flows  across  a  wide  range  of  transmission  paths  in  an  interoperable 
manner  with  naval,  joint,  coalition  and  civil  /  law  enforcement  agencies.  Platforms  and 
vehicles  are  able  to  communicate  freely  and  autonomously  with  other  elements  of  the 
architecture  thus  the  existence  and  functions  of  the  underlying  network  are  transparent 

to  the  warfighter. 

Attribute 

Notional  Metric 

Capacity 

Throughput  (1)  effective  systems  capacity  =  maximum  data  rate  - 
system  overhead  rate  (2)  bandwidth  utilization  =  available  data  rate  / 
effective  systems  capacity 

Reach 

Percentage  of  nodes  that  can  communicate  in  desired  access  modes, 
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information  formats,  and  applications 

Connectivity 

Percentage  of  time  that  all  required  nodes  are  connected  to  the 
network 

Information 

Assurance 

Extent  to  which  node  supports  the  assurance  of  information  in  the 
areas  of  privacy,  availability,  integrity,  authenticity,  and  non¬ 
repudiation 

Quality  of 

Service 

Measures  of  jitter,  packet  loss  and  latency 

Timeliness 

Degree  (speed  of  effect)  to  which  currency  matches  what  is  needed 
(0=no  match,  l=high  degree  of  matching  between  currency  level 
needed  and  available) 

Agility 

Extent  to  which  the  network  can  maintain  QOS  in  response  to 
environmental  changes  (incorporates  robustness,  responsiveness, 
flexibility,  innovativeness  and  adaptation) 

Robustness 

Number  of  differing  conditions/environments  over  which  network  is 
capable  of  operating  at  a  given  level  of  effectiveness  (baseline  level 
determined  by  SME,  simulation,  analysis,  empirical  analysis,  etc.) 

Effectiveness  of  network  across  varying  levels  of  attack/degradation 
(baseline  level  determined  by  SME,  simulation,  analysis,  empirical 
analysis,  etc.) 

Number  of  tasks/missions,  which  the  network  is  capable  of  operating 
at  a  given  level  of  effectiveness  (baseline  level  determined  by  SME, 
simulation,  analysis,  empirical  analysis,  etc.) 

Responsiveness 

The  timeliness  of  the  response  to  an  environmental  change  (baseline 
level  determined  by  SME,  simulation,  analysis,  empirical  analysis, 
etc.) 

Flexibility 

Number  of  options  for  responding  to  an  environmental  change 

Compatibility  of  different  responses  (0=not  compatible,  l=fully 
compatible;  determined  by  SME,  simulation,  analysis,  empirical 
analysis,  etc.) 

Innovativeness 

Number  of  novel  responses  developed  and  implemented  (baseline 
determined  by  SME,  simulation,  analysis,  empirical  analysis,  etc.) 

Adaptiveness 

Number  and  timeliness  of  changes  to  network  structure  and  processes 
(baseline  determined  by  SME,  simulation,  analysis,  empirical 
analysis,  etc.) 

4.  Provide  adaptive  /  automated  decision  aids:  To  support  warfighter  decision  making 
by  providing  recommended  courses  of  action  that  are  adaptive  and  based  upon 
knowledge  of  the  operational  context,  commander’s  intent,  rules  of  engagement,  order  of 

battle,  etc,  and  evolution  of  the  battlespace  landscape _ 

Attribute _ Notional  Metric _ 

Robustness _  Degree  to  which  decision  aids  support  decision  making  across  a  range 
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of  situations  and  degradation  conditions 

Responsiveness 

Degree  to  which  decision  aids  support  decision  making  which  is 
relevant  and  timely 

Innovativeness 

Degree  to  which  decision  aids  support  decision  making  that  reflects 
novel  ways  to  perform  known  tasks 

Adaptability 

Degree  to  which  decision  aids  support  a  decision  making  process  with 
the  flexibility  to  alter  decision  making  in  response  to  the  evolution  of 
the  battlespace  landscape 

Consistency 

Extent  to  which  decision  aids  support  decision  making  are  internally 
consistent  with  prior  understanding  and  decisions 

Currency 

Extent  to  which  decision  aids  support  decision  making  that  minimizes 
latency  (e.g.  Notification  -  Time  of  detection  =  Cueing  Time,  Time  of 
detection  -  receipt  of  refined  positional  estimate  =  Update  rate,  Time 
of  cueing  data  -  time  of  weapon  firing  =  weapons  release  time,  Firing 
report  received  by  group  commander  -  weapons  firing  time  =  Firing 
report  time) 

Precision 

Error  and  confidence  level  for  time  and  position  information 
compared  to  a  standard  reference 

Fitness  for  Use 

Relative  quality  in  reference  to  criteria  that  are  determined  by  the 
situation 

Appropriateness 

Extent  to  which  decision  aids  support  decisions  that  are  consistent 
with  existing  understanding,  command  intent  and  values 

Completeness 

Extent  to  which  decision  aids  support  relevant  decisions  that 
encompass  the  necessary: 

•  Depth:  range  of  actions  and  contingencies  included 

•  Breadth:  range  of  force  elements  included 

•  Time:  range  of  time  horizons  included 

5.  Provide  human-cen 

through  all  phases  of  w 
Interfaces  which  dynan 
best  leverage  the  relath 
uncertain/ambiguous  ci 
speed  complex  calculal 

trie  integration:  Enhance  the  ability  of  warriors  to  multi-task 
arfare  while  taking  advantage  of  improved  Human-Computer 
lically  assign  function  to  human  and  information  systems  that 
/e  strengths  of  each  (e.g.,  human  decision  making  in 
rcumstances,  computer  systems  in  situations  relying  upon  high 
ions). 

Attribute 

Notional  Metric 

Competence 

Distribution  of  members’  knowledge,  skills,  abilities  and 
attitudes. 

Trust 

Extent  to  which  members  are  willing  to  rely  on  one  another 

Confidence 

Extent  to  which  members  have  expectations  of  the  reliability  of 
the  organization 

Size 

Number  of  team  members  involved  adequate  to  support 
mission 
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Experience 

Degree  to  which  team  members  have  interacted  in  the  past  on 
the  same  task 

Diversity 

Degree  to  which  team  members  are  heterogeneous  or 
homogeneous  across  exogenous  variables:  experience,  age, 
gender,  etc. 

Autonomy 

Extent  to  which  organization  is  externally  or  self  directed 

Structure 

•  Numbers  of  layers  of  authority 

•  Functional  Differentiation  Effectiveness 

Interdependence 

Extent  to  which  members  depend  on  one  another  for  resources 

Cooperation 

Extent  to  which  member(s)  are  willing  and  able  to  work 
together 

Efficiency 

Extent  to  which  members  utilize  one  another’s  resources  so  as 
to  minimize  costs  and  maximize  benefits 

Synchronization 

Extent  to  which  organization  is  conflicted,  deconflicted,  or 
synergistic 

Engagement 

Extent  to  which  all  members  actively  and  continuously 
participate 

Risk  Propensity 

Extent  of  risk  aversion 

6.  Provide  information  weapons:  To  integrate  the  use  of  military  deception, 
psychological  operations,  electronic  warfare,  and  physical  destruction,  mutually 
supported  by  intelligence,  in  order  to  deny  information,  influence,  degrade,  or  destroy 
adversary  information,  information-based  processes,  and  information  systems.  (Metrics 
are  under  development.) 

Attribute 

Notional  Metric 

Lethality 

Extent  of  capability  to  precisely  deliver  desired  Non-Kinetic 
(NK)  Information  Operations  (10)  effects. 

Coverage 

Extent  of  capability  to  accomplish  10  effects. 

Persistence 

Extent  of  capability  to  sustain  10  effects. 

Timeliness 

Extent  of  capability  to  deliver  desired  NK  10  effects  at  a 
desired  time. 

Survivability 

Extent  of  capability  to  avoid  enemy  threats,  counter  ISR,  and 
employ  10  techniques  to  reduce  targeting  of  adversary  kinetic 
systems  allowing  increased  secure  maneuvering  by 

ASMD/Deny  ISR/SEAD/Networks. 

Table  1  FORCEnet  Capability  Descriptions,  Attributes  and  Metrics 

To  define  where  measures  may  be  applied  to  evaluate  understanding,  key  factors  and 
attributes  must  be  organized.  Here  the  individual-cognitive  and  group-collaborative 
relationships  dominate.  It  is  largely  a  process  analysis  addressing  critical  cognitive 
processes  and  performance  issues  in  terms  of  individuals,  aggregated  teams,  and 
organization.  Collaboration  among  individuals,  formal  and  informal  organizations,  and 
ad  hoc  physical  and  virtual  groups  are  central  to  the  analysis.  The  types  of  interactions 
(e.g.,  information  sharing,  course  of  action  analysis,  decision  making)  that  need  to  be 
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supported,  a  range  of  preferred  means  of  interaction  (e.g.,  “face  to  face,”  document 
exchange,  graphical  reference),  the  frequency  of  desired  interaction,  the  range  of  actors 
needed  in  collaboration  (e.g.,  intra-echelon,  inter-echelon,  joint,  non-military),  and  the 
reasons  for  these  interactions  (e.g.,  emotion,  de-confliction,  affirmation,  assessment) 
must  all  be  addressed.  Measures  of  interest  are  those  that  address  how  people  perceive 
and  act  on  information,  and  what  information  they  need  at  both  the  individual  and  group 
levels.  Systems  performance,  interoperability,  information  flow  and  other  measures  also 
pertain. 

To  define  measures  to  assess  influence  and  ability  to  effect  desired  outcomes  is  largely  an 
operations  analysis  problem.  Here  the  other  two  analyses  are  aggregated  to  answer  the 
question:  Were  desired  operational  effects  and  goals  met?  This  analysis  should  provide 
insights  into  how  well  human  collaboration  functioned  using  the  available  technical 
infrastructure.  It  should  also  support  an  understanding  of  how  well  the  human-machine 
interface  worked.  Criteria  for  assessment  and  optimization  goals  should  be  generated  in 
order  to  apply  measures  of  performance  and  effectiveness.  These  measures  may  address 
complexity  issues  associated  with  the  number  of  nodes  involved  and  timeliness  of 
information  exchange  to  meet  specific  operational  objectives  as  well  as  aggregated 
objectives.  They  may  also  address  effectiveness  in  terms  of  synchronization  of  activities 
in  time  and  space. 

To  define  measures  to  assess  action  is  largely  a  resource  analysis  problem  that  depends 
on  the  first  two  analyses  while  focusing  on  the  performance  of  the  technical 
infrastructure.  It  should  address  specifically  the  ability  of  information  technology  and 
communications  systems  (including  sensors,  visualization,  and  collaborative  tools)  to 
exchange  needed  data  and  the  information  it  conveys.  Latencies,  bandwidth,  connectivity, 
and  numerous  other  technical  measures  of  performance  can  be  identified.  This  analysis 
should  also  provide  insights  into  how  well  technical  capabilities  can  be  maintained  in 
degraded  environments,  “workarounds,”  configuration  issues,  interoperability,  and 
“fusion”  of  data  feeds.  Insights  into  usability  issues  associated  with  workstation 
operations,  applications,  and  ergonomic  issues  also  might  be  investigated  here.  From  a 
C4ISR  perspective,  the  water’s  edge  is  achieving  the  ability  to  allocate  resources 
including  weapons  and  the  ability  to  respond  and  adapt  infrastructure  in  the  face  of 
changing  circumstances. 

Together  these  analyses  should  provide  a  blended  view  of  system  functionality.  But  they 
are  insufficient  without  bounding  the  analysis  through  the  analytical  framework  discussed 
earlier:  perspective,  knowledge  management  processes,  operational  scope,  and  network 
centric  “domains.” 

Additional  measures  that  adapt  traditional  approaches  to  a  new  environment  also  should 
be  considered.  An  example  may  be  “recognition  differential” — the  ability  of  an 
individual  or  team  to  note  a  change  in  the  environment  and  its  implications.  This  may  be 
useful  in  assessing  information  overload  and  propagation  paths  for  this  condition.  Such  a 
recognition  differential  may  reflect  latencies  in  another  attribute,  ,  relating  the  time 
information  is  available  to  the  time  it  is  used  as  well  as  differences  in  subjective 
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interpretation.  Information  overload  may  be  assessed  in  terms  of  individual  indicators 
(e.g.,  human  performance),  technological  indicators  (such  as  bandwidth  and  data  flow), 
and  “decisiveness”  as  a  result  of  a  decision  maker  having  sufficient  information  to  make 
a  decision. 

Propagation  paths  for  key  information  also  will  relate  to  identifying  the  means  for 
mitigating  and  taking  advantage  of  complexity.  A  path  approach  should  also  include  the 
number  of  nodes  required  for  a  “decisive”  interaction.  It  also  may  include 
“amplification,”  that  is  needs  driven  by  non-linear  aspects  of  complex  interactions  that 
increase  or  decrease  the  speed  of  decision  making — perhaps  in  the  form  of  errors  or  rate 
of  gaining  shared  awareness. 

Additional  measures  may  include  dispersion,  vulnerability,  risk,  readiness,  and  number  of 
decision  elements.  Dispersion  may  be  driven  by  time-space  separation  of  actors  and 
include  such  things  as  how  long  it  takes  to  reach-back  and  integrate  key  subject  matter 
experts  in  analytical  and  decision  support  functions.  Vulnerability  may  be  addressed  in  a 
range  of  dimensions  to  include  error  propagation,  trust  in  information  and  sources, 
effects-based  “windows”  of  decision-making,  and  more  traditional  assessments  regarding 
information  assurance  and  the  operational  impact  of  lost  information  inputs.  For  example, 
risk  relates  closely  to  vulnerability  and  may  be  assessed  as  a  function  of  the  interaction  of 
key  vulnerability  and  threat  factors  at  any  given  point  in  time.  Readiness  may  be 
determined  by  the  availability  and  timely  use  of  organically  and  non-organically 
controlled  assets  in  terms  of  key  points  in  time.  Last,  the  number  of  decision  elements 
may  be  assessed  to  indicate  the  complexity  and  sufficiency  of  information  requirements 
across  a  range  of  analytical  dimensions. 

The  metrics  applied  in  the  analyses  and  assessment  phase  will  be  dependent  upon  the 
assumptions  that  define  them.  They  should  provide  emphasis  on  quantification  of 
performance  and  effectiveness.  These  measures  and  metrics  may  be  key  to  assessing  an 
objective  state  of  shared  awareness.  While  the  measures  and  metrics  described  above 
provide  a  set  of  potentially  useful  “types,”  specific  measures  to  be  applied  should  be 
drawn  from  a  consolidated  pool  of  existing  and  evolving  activities  in  the  naval  and  joint 
community. 


Performance  Based  Measurement:  Situation  Awareness  (SA) 

Individuals,  organizations  and  teams  can  each  be  regarded  as  an  independent  cognitive 
system.  Where  individuals  collaborate  inter/intra  organizationally,  often  in  a  ubiquitous 
and  virtual  workspace,  they  aggregate  their  cognition  into  shared  situation  awareness 
(SSA).  Multiple  channels  permit  the  storage  and  mining  of  information  that  can  be 
discovered,  acquired,  shared  and  leveraged  at  each  level.  First,  individuals  develop  their 
independent  SA.  The  second  element  of  the  process  is  the  effective  communication  of 
individual  perceptions  to  the  other  members  of  the  group,  for  the  purpose  of  consensus 
building.  Third  is  the  subjective  aggregation  of  those  inputs  by  team  members.  Last  is 
objective  visualization  of  a  common  base  of  perception,  often  in  visualization  tools.  In 
this  process,  emerging  technologies  for  decision  aids  like  intelligent  agents,  data  mining 
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and  complex  modeling  offer  the  potential  for  large  volumes  of  data  to  be  collected, 
processed,  and  displayed  without  overloading  users.  Correlated  data  becomes  information 
that  is  converted  into  knowledge  which  relies  upon  accurate  situation  awareness. 

The  expected  result  of  leveraging  virtual  interactions  without  the  barriers  of  time  or  space 
should  be  a  more  agile  and  responsive  organization.  More  fundamentally,  information 
should  be  sufficiently  available  that  the  network  should  naturally  foster  collaboration. 
However,  the  analytic  space  is  not  clearly  bounded.  The  challenge  in  the  analysis  process 
is  that  network  centric  warfare  explicitly  involves  the  human  component  and  the  other 
issues  that  arise  from  the  complex  interactions  of  distributed  teams.  These  teams  create  a 
multi-dimensional  analytic  space  that  includes  tightly  connected  interactions  between 
platforms,  systems  and  people.  Often  these  interactions  are  subjective  and  therefore 
measures  of  effectiveness  are  impacted  by  elements  such  as  organizational  development 
and  culture,  morale,  doctrine,  training  and  experience. 

SA  “quality”  then  is  the  degree  to  which  SA  accurately  comprises  an  objective 
assessment  of  that  reality,  an  assessment  that  can  be  shared  and  then  interpreted  for 
application  in  a  given  individual’s  context.  The  difference  between  knowledge-based  and 
performance-based  techniques  for  evaluating  situation  awareness  reflects  taking 
measurements  at  different  points  in  the  process  of  user  cognition.  Performance-based 
measurements  have  been  determined  to  ascertain  the  timing  and  substance  of  a  user's 
reaction  to  realistic  situations,  while  knowledge  based  techniques  are  more  appropriate 
for  providing  a  detailed,  theoretical  assessment  of  the  subject's  situation  awareness.4 
Performance-based  measurement  is  complementary  to  knowledge -based  measurement  in 
the  human  factors  analysis.  Based  on  the  best  current  practices  in  fields  such  as  human 
resources  and  organizational  development,  Capability  Maturity  ModelSM  (CMM)  based 
approaches  often  help  organizations  to  characterize  the  maturity  of  their  work-force 
practices,  guide  a  program  of  continuous  work-force  development,  set  priorities  for 
immediate  actions,  integrate  work-force  development  with  process  improvement,  and 
establish  a  culture  of  excellence.5  This  approach  facilitates  the  evolutionary  improvement 
path  from  ad  hoc,  inconsistently  performed  practices,  to  a  mature,  disciplined 
development  of  the  knowledge,  skills,  and  motivation  of  the  work  force. 

We  suggest  a  variant  of  the  CMM  to  support  SA  maturity  assessment.  This  variant 
consists  of  five  maturity  levels  that  institutionalize  a  level  of  capability  for  nurturing  the 
talent  within  the  organization,  developing  effective  teams,  and  successfully  managing  the 
people  assets  of  the  organization.  The  benefit  of  CMM  to  the  notion  of  SA  is  that  the 
CMM  guides  an  organization  in  implementing  a  series  of  increasingly  sophisticated 
practices  and  activities  for  developing  and  motivating  its  workforce  which  can  have  a 
significant  impact  on  individual,  team,  unit,  and  organizational  performance. 


4  Pritchett,  A.  R.  H.,  R.J.  &  Johnson,  E.N.  (1996).  Use  of  Testable  Responses  for  Performance-Based 
Measurement  of  Situation  Awareness.  Presented  at  the  International  Conference  on  Experimental  Analysis 
and  Measurement  of  Situation  Awareness,  Daytona  Beach,  FL 

5  Harigopal,  U.  S.,  Antony,  (2001).  “Cognizant  Enterprise  Maturity  Model  (CEMM).”  IEEE  Transactions  on  Systems, 

Man,  and  Cybernetics:  Special  Issue  on  Knowledge  Management. 
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Level 


Focus 


Process 


5 

Transformational 

Agent  based 
communication; 
establishing  a  process 
for  adapting  processes 
to  support  operational 
contingencies; 
establish  knowledge 
delivery  mechanism 
to  provide  knowledge 
to  strategic  partners; 
process  optimization 

Evaluation  of  performance  and 
effectiveness  on  a  continuous  basis 

Identify  adjustments  and  potential 
improvement  to  the  fusion  process 
Determine  source  specific  data 
requirements  for  processing 

Recommend  allocation  and  direction  of 
resources  in  support  of  the  mission 
Understand  mission,  opportunities  and 
risks,  adversary’s  capabilities  and 
limitations,  analysis  of  possible 
outcomes,  and  adversary’s  intent 

4 

Predictable 

Concepts  embedded 
in  data  translated  into 
a  common  ontology; 
data  mining  for 
patterns  and 
relationships; 
presentation  of 
knowledge  based 
upon  the  user’s 
learning  profile; 
network  of  multiple 
portals  enables  the 
real-time  aggregation 
of  disparate 
knowledge 

Estimate  capabilities,  i.e.  number  and 
location 

Predict  enemy  intent  based  on  actions, 
communications  and  enemy  doctrine 
Identify  threat  opportunities  -  ID  of 
potential  opportunities  for  enemy  threat 
Assess  from  multi-perspectives 

Analyze  prediction  of  offensive/defensive 
results  of  hypothesized  engagements 
Understand  mission,  opportunities  and 
risks,  adversary’s  capabilities  and 
limitations,  analysis  of  possible  outcomes 
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Level 


Focus 


Process 


3 

Defined 

Data  is  aggregated  in 
a 

central  data  base; 
data  from  multiple 
operational  systems 
can  be  extracted  on 
demand; 

richer  artifacts  of  the 
process  are  stored  and 
organized;  data 
presentation  includes 
summaries  and 
analysis; 

collaborative  tools 
capture  the  timeliness, 
breadth  and  depth  of 
subject  matter  experts 

Estimate  relationships  among  aggregated 
objects  including  events/activities 

Interpret  within  context  weather,  terrain 
and  other  environmental  considerations 
Assessment  from  a  multi-perspective  (i.e. 
Blue,  Red  &  White  viewpoints) 

Understand  mission,  opportunities  and 
risks,  adversary’s  capabilities  and 
limitations 

2 

Repeatable 

Data  repository 
mechanism  provided 
to  capture  individual 
input  and  retrieve 
data;  forum  provided 
for  distributed 
collaboration 

Focus  on  individual  objects 

Associate  sensor  outputs  w/specific 
known  objects  or  initiate  new  objects 

Use  sensor  data  to  refine  the  best 
estimates  of  current  positions  for  each 
hypothesized  object. 

Understand  mission,  opportunities  and 
risks 

1 

Initial 

Limited  collaboration, 
data  fusion  or 
correlation 

Align  data  with  respect  to  time/space 
Relate  newly  received  observations  to 
existing  track 

Comprehend  basic  classification  of 
emitters,  platforms,  etc. 

Understand  mission 

Table  2  Five  Levels  of  Situational  Awareness 

Level  1:  The  Initial  Level 

At  the  initial  level,  the  organization  typically  does  not  provide  a  stable  environment. 
During  a  crisis,  planned  procedures  are  abandoned.  Success  depends  entirely  on  having 
an  exceptional  leader.  Even  strong  tactics,  techniques  and  procedures  (TTP)  cannot 
overcome  the  instability.  Capabilities  of  Level  1  organizations  are  typically  unpredictable 
because  the  process  is  ad  hoc  and  occasionally  chaotic.  Few  processes  are  defined. 
Performance  depends  on  the  capabilities  of  individuals  and  varies  with  their  innate  skills, 
knowledge,  and  motivations.  Therefore  performance  can  be  predicted  only  in  terms  of 
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individual  rather  than  organizational  capability.  This  is  the  lowest  level  of  data  fusion  and 
often  that  fusion  does  not  exist.  Without  any  data  fusion,  SA  is  based  on  the  manual 
correlation  and/or  aggregation  of  data.  This  would  also  correspond  to  the  lowest  level  of 
individual  SA,  which  is  the  fundamental  perception  of  important  information. 

Level  2:  The  Repeatable  Level 

Policies  and  procedures  are  established  and  institutionalized.  Planning  and  managing  new 
tasks  are  based  on  experience  with  similar  projects,  which  allow  organizations  to  repeat 
successful  practices  developed  in  earlier  tasks.  An  effective  organization’s  process  is 
practiced,  documented,  enforced,  trained,  measured,  and  able  to  improve.  Problems  in 
meeting  goals  and  performance  standards  are  identified  when  they  arise.  Level  2 
organizations  can  be  summarized  as  disciplined  because  planning  and  execution  of  the 
mission  is  stable  and  earlier  successes  can  be  repeated.  The  key  process  areas  at  Level  2 
focus  on  instilling  basic  discipline  into  workforce  activities.  From  the  standpoint  of  data 
fusion  the  focus  is  on  individual  objects.  SA  as  a  construct  is  still  fundamentally  about 
basic  perceptions  of  important  information. 

Level  3:  The  Defined  Level 

The  Level  3  organization  exploits  effective  policies  and  procedures  that  are  well 
documented  and  integrated  into  a  coherent  whole.  There  is  a  dedicated  component 
organization  that  has  been  institutionalized  and  is  responsible  for  the  organization's 
process  activities,  i.e.  quality  control/analysis.  An  organization-wide  training  program  is 
implemented  to  ensure  that  the  staff  and  managers  have  the  knowledge  and  skills 
required  to  fulfill  their  assigned  roles.  A  well-defined  process  can  be  characterized  as 
including  readiness  criteria,  inputs,  standards,  and  procedures  for  performing  the  work, 
verification  mechanisms  (such  as  peer  reviews),  outputs,  and  completion  criteria. 
Because  the  process  is  well  defined,  management  has  good  insight  into  the  level  of 
performance  that  is  based  on  a  common,  organization-wide  understanding  of  activities, 
roles,  and  responsibilities.  The  key  process  areas  at  Level  3  are  knowledge  and  skills 
analysis,  workforce  planning,  competency  development,  career  development, 
competency-based  practices,  and  participatory  culture.  Data  fusion  is  devoted  to 
organizing  the  hypothesized  objects  into  a  big  picture  of  what  is  happening.  The  big 
picture  is  described  in  terms  of  groups  or  organizations  of  objects  so  that  decisions  can  be 
made  by  decision  makers  about  how  to  use  friendly  organizations.  SA  goes  beyond 
perception  and  encompasses  the  combining,  interpreting,  storing  and  retention  of 
information.  At  this  level  of  SA,  operationally  relevant  meaning  and  significance  of  the 
Level  2  data  is  being  considered. 

Level  4:  The  Predictable  Level 

The  organization  sets  quality  goals  that  are  measured  as  part  of  an  organizational 
measurement  program.  Processes  are  instrumented  with  well-defined  and  consistent 
measurements.  Organizational  control  over  performance  relies  upon  narrowing  the 
variation  in  performance  to  fall  within  acceptable  quantitative  boundaries.  Meaningful 
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variations  in  process  performance  can  be  distinguished  from  random  fluctuations.  The 
performance  of  Level  4  organizations  is  predictable  because  performance  is  measured 
and  operates  within  measurable  limits.  These  measurements  permit  an  organization  to 
predict  trends  in  process  quality  and  when  the  quantitative  bounds  of  these  limits  are 
exceeded,  action  is  taken  to  correct  the  situation.  The  key  process  areas  at  Level  4  focus 
on  mentoring,  team  building,  team-based  practices,  organizational  competency 
management,  and  organizational  performance  alignment.  At  this  level  the  data  fusion  is 
more  about  the  situation  and  what  is  known  from  enemy  doctrine  and  objectives  to 
predict  the  strength  and  vulnerabilities  for  both  the  threat  and  friendly  forces.  Almost  at 
the  highest  level  of  SA,  there  is  some  capability  to  forecast  future  situation  and  events. 
Given  a  high  level  of  understanding  of  the  situation  future  events  and  their  implications 
to  permit  timely  decision-making. 

Level  5:  The  Transformational  Level 

At  Level  5  the  organization  is  focused  on  continuous  process  improvement.  The 
organization  identifies  weaknesses  and  strengths  proactively,  with  the  goal  of  preventing 
the  occurrence  of  negative  performance.  Innovations  that  exploit  best  practices  are 
identified  and  transferred  throughout  the  organization.  Level  5  organizations  analyze 
defects  to  determine  their  causes.  Level  5  organizations  are  continuously  striving  to 
improve  the  range  of  their  process  capability,  thereby  improving  their  performance. 
Improvement  occurs  both  by  incremental  advancements  in  the  existing  process  and  by  the 
introduction  of  innovations.  The  key  process  areas  at  Level  5  address  continuous 
improvement  for  personal  competency  development,  coaching,  and  workforce 
innovation.  The  fusion  process  at  this  level  examines  what  is  unknown  in  the  context  of 
the  situation  and  threat  and  then  develops  options  for  collecting  the  information. 

At  the  highest  level  of  SA,  relying  more  on  tacit  communications,  organizations  are  self- 
synchronized  and  are  heavily  dependent  on  future  predictions. 

Evidence  to  date  suggests  that  the  predictability,  effectiveness  and  control  of  processes 
improve  as  the  organization  moves  up  the  five  levels  of  CMM.  This  offers  insight  into 
addressing  a  major  organizational  challenges  to  transformation  in  the  Department  of 
Defense  that  often  dominate  the  technological  challenges.  As  rapid  advances  in 
information  technology  enable  network  centric  warfare  to  move  from  concept  to  the 
battlesphere,  traditional  metrics  need  to  be  reevaluated.  Performance  improvements  such 
as  increased  responsiveness  and  efficiency  need  to  be  measured  in  the  context  of 
Situation  Awareness  and  are  fundamental  to  guiding  process  improvements  in  the  storing, 
organizing  and  processing  of  information.  CMM  is  a  useful  framework  for  the 
measurement  of  SA  and  provides  a  performance  based  assessment  of  the  value  of  C4ISR. 

Summary 

In  this  paper,  we  have  provided  a  foundation  for  an  assessment  framework  involving 
C4ISR  processes  in  FORCEnet.  This  framework  reflects  and  expands  upon  work  done  by 
a  number  of  organizations  engaged  in  efforts  to  structure  a  process  that  links  traditional 
and  evolved  C4ISR  attributes,  measures,  and  metrics  to  network  centric  outcomes 
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including  work  being  done  in  support  of  the  Office  of  Force  Transformation  and  the 
office  of  the  Assistant  Secretary  of  Defense  for  Networks,  and  Information  Integration 
(ASD  (NI2)).  Using  a  capabilities  based  approach  to  analysis,  metrics  have  been 
recommended  and  developed  to  measure  organizational  performance.  Performance 
improvements  such  as  increased  responsiveness  and  efficiency  have  been  described  in  the 
context  of  Situation  Awareness  (SA),  and  in  a  form  consistent  with  the  framework  of  a 
Capability  Maturity  Model.  Finally,  we  have  provided  five  levels  of  Shared  Situational 
Awareness  that  can  provide  the  basis  for  assessing  an  organization  in  a  specific  domain 
relevant  to  FORCEnet.  Hopefully,  continued  work  will  result  in  implementing  a  series  of 
increasingly  sophisticated  practices  and  activities  which  can  have  a  significant  impact  on 
individual,  team,  unit,  and  organizational  performance. 
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Why  Measure  Value? 
Budgeting  for  C4ISR  Operational  Effect 

•How  much  weapon  system  automation  exists? 


Enduring  Drivers  of  C4ISR  Value 

Assessment 


Ability  to  sense 

-  through  globally  distributed  technological-cognitive  systems; 

Ability  to  understand 

-  individual  perception,  shared  awareness,  and  aggregation  in 
organizations  to  result  in  knowledge  mobilization; 

Ability  to  influence 

-  friend,  foe,  and  others  by  conditioning  of  the  operational  space 
through  behaviors,  lines  of  authority,  positioning, 

Ability  to  act  rapidly,  decisively,  and  appropriately 

-  as  coordinated  across  tactical,  operational,  and  strategic  echelons; 

Ability  to  visualize 

-  the  operational  space  in  ways  relevant  to  individual  users; 

Ability  to  effect 

-  desired  outcomes. 


FORCEnet:  Vision  to  Reality 
A  Holistic  Approach* 


FORCEnet  Vision  & 

Strategy  *  *  * 

*  Enable  domination  across  full 
spectrum  of  warfare 

*  Develop  New  Operational 
Concepts  out  to  2020 

*  Focus  Technologies  for 
Enhanced  Capabilities 

*  Develop  Broad  Program  & 
Resource  Plans 


FORCEnet  Functional 
Objectives 


•  Decision  Superiority 

•  Increased  Options 

•  Increased  Agility 

•  Improved  Coordination 


Navy  Leaders 
make  appropriate 
course  corrections 


FORCEnet 

Requirements 

•  Mission  Capability 
Packages  /  Naval 
Capability  Studies 

*  Human*  Capability 
Packages 


Sea  Shield 


Sea  Trial 


> 


Sea  Warrior 


Sea  Enterprise 


^*Sea  Strike 


Sea  Basing 


*  Knowledge  Mobilization 


*  Human  Capability  Packages  -  a  means  to  achieve  mission  capability  -  by  equipping  the  man 

*FROM  NDIA  FORCEnet  Study  Human  Element  Group  Report. 

STRENGTH  THROUGH  INDUSTRY  &  TECHNOLOGY 


FORCEnet  C4ISR  “Tradespace 


* 


Analytical  Dimensions 

•  Perspective  (individual, 
organizational,  and  systems). 

•Knowledge  management 
processes  (cognitive, 
collaborative,  and 
integrative). 

•Operational  scope  (tactical, 
operational,  and  strategic). 


•Network  centric  “domains” 
(cognitive,  information,  and 
physical). 

FORCEnet  Objectives 

•  Decision  Superiority 

•  Increased  Options 

•  Increased  Agility 

•  Improved  Coordination 


Knowledge  Mobilization 


*Adapted  from  NDIA  FORCEnet  Study  Human  Element  Group  Report. 


FORCEnet  Case  Study 


•  Framework  for  Measuring  C4ISR 

•  Relationship  to  the  POM  Process 

•  Functionally  Decomposing  Capabilities 

•  Measurements  and  Attributes 

•  Measuring  Shared  Situational  Awareness 

•  Human  Centric  Experimentation 


Infosphere  Vision 


Decomposing  the  Infosphere 


ATTACKING 


Guidance 

LOCATING 


Measured  Transformation  - 
A  Capabilities  Driven  Process 


•  For  our  purposes,  we  define  a  capability  as: 

-  that  combination  of  human,  technological,  organizational, 
process,  and  cognitive  elements  that  provides  the  means  to 
achieve  a  clearly  articulated  outcome  in  a  defined  context. 

•  Implications: 

-  Emphasizes  ability  to  link,  delink,  and  relink  these  elements  in 
ways  that  are  tailored  to  unique  circumstances 

-  Embeds  the  human  individually  and  as  part  of  an  organization 
in  a  complex  system  of  systems 

-  Accepts  intangible  cognitive  processes  as  a  leveraged  part  of 
the  system. 

-  Establishes  emphasis  no  longer  on  the  “who”  (i.e.  traditional 
threat  based  approach)  but  rather  on  the  “how”. 


Attributes,  Measures,  &  Metrics 


•  Attribute:  some  aspect  of  an  event,  situation,  person,  or 
object  considered  important  to  understanding  the  subject 
under  study 

•  Measure:  a  vehicle  or  standard  by  which  some  attribute  of 
interest  is  recorded 

•  Metric:  the  application  of  a  measure  to  two  or  more  cases 
or  situations  to  derive  the  mathematical  dimensions, 
capacity,  and  amount  of  some  thing,  process,  or  effect 

•  Based  on 

-  ASD(C3I)/OFT  Framework  for  NCW 

-  CCRP  and  NATO  Codes  of  Best  Practice  (C2  Assessment  & 
Experimentation) 

-  Others 


A  Hierarchical  Capability  Taxonomy 

Defined  FORCEnet  Capabilities 


1 .  Provide  expeditionary,  multi-tiered  sensor  and  weapon 
information 

2.  Conduct  distributed,  collaborative  command  and  control 

3.  Provide  dynamic,  multi-path  and  survivable  networks 

4.  Provide  adaptive/automated  decision  aids 

5 .  Provide  human-centric  integration 

6.  Provide  information  effects 


Sea  Power  21  and  Naval  Transformation  Roadmap 


Current  Top-Level  FORCEnet  Capabilities/Requirements 


Collect,  Process 
and  Distribute 
Organic  sensor  and 
weapon  Information 


Collect,  Process 
and  Distribute 
Non-organic  Sensor 
Information 


•  Capabilities-Based 
Requirements/Acquisition 

•  Analysis/Assessment 

•  Technology  Transition 

•  Experimentation/Innovation 

•  Refined/updated  as  part  of  an  iterative, 
lessons-leamed  process 


1.  Provide  expeditionary,  multi-tiered  sensor  and  weapon  information 

The  expeditionary,  multi-tiered  sensor  and  weapons  grid  capability  uses  a  full  spectrum  of  manned  and 
unmanned  vehicles,  platforms,  sensors  and  weapons  to  provide  the  Force  Commander  with  what  is  needed  to 
locate  targets  and  attack  them  across  the  depth  and  breadth  of  a  theater-sized  battlespace.  Sensors  must 
determine  their  position,  time  and  movement  at  the  precise  time  they  are  reporting  their  target  or  other 
intelligence  information.  The  time  and  position  information  of  the  track  provided  by  sensors  in  the  grid  must  be 
properly  attributed  (e.g.,  linked  to  a  standard  reference  frame  with  uncertainty  (error)  and  confidence  level)  for  it 
to  be  accurately  understood,  represented  and  fused  with  other  data  /  information.  Many  modem  weapons  are 
also  dependent  on  precise  time  and  position  (including  uncertainty)  for  effective  operation. 

Attribute 

Notional  JVlcasurcs 

Accuracy 

Correspondence  with  ground  truth-correlation  coefficient  (0  =  no 
correspondence  with  ground  truth,  1  =  full  correspondence  with  ground 
truth).  Data  matrix  comprised  of  relevant  information  items  estimates  (for 
instance:  detection,  ID,  velocity,  location,  heading,  etc.) 

Consistency 

Degree  of  lack  of  ambiguity  with  previous  information 

Completeness 

Percentage  of  ground  truth  relevant  and  necessary  for  ongoing  task 

Precision 

Error  and  confidence  level  for  time  and  position  information  compared  to  a 
standard  reference 

Timeliness 

Degree  to  which  currency  matches  what  is  needed  (0  =  no  match,  1  =  high 
degree  of  matching  between  currency  level  needed  and  available) 

2.  Conduct  distributed,  collaborative  command  and  control 

To  collaboratively  manage  land,  air,  sea,  and  space  operational  forces  in  time,  space,  and  purpose  to  produce 
maximum  relative  combat  power  and  minimize  risk  to  own  forces.  This  activity  ensures  all  elements  of  the 
operational  force,  including  supported  agencies’  and  nations’  forces,  are  efficiently  and  safely  employed  to 
maximize  their  combined  effects  beyond  the  sum  of  their  individual  capabilities. 

Attribute 

Notional  Measures 

Shared  Situational 
Awareness 

Degree  to  which  the  different  individual  mental  models  of  the  situation 
are  integrated  into  a  common  operational  picture. 

Quantity  of  Posted 
Information 

Percent  of  collected  information  posted 

Quantity  of 
Retrievable 
Information 

Percentage  of  nodes  that  can  retrieve  various  sets  of  information. 

Understandability 

Degree  to  which  information  is  easy  to  use  (0  =  low  degree  of  ease  of 
use,  1  =  high  degree  of  ease  of  use) 

Precision 

Error  and  confidence  level  for  time  and  position  information  compared  to 
a  standard  reference 

Timeliness 

Degree  (speed  of  effect)  to  which  currency  matches  what  is  needed  (0  = 
no  match,  1  =  high  degree  of  matching  between  currency  level  needed 
and  available) 

3.  Provide  dynamic,  multi-path  and  survivable  networks  (2/2) 

To  provide  data  and  information  flow  seamlessly  and  transparently  to  the  warfighter  across  a  fault  tolerant, 
adaptable,  self-organizing,  holistically  engineered  continuously  available  network.  The  data  and  information 
flows  across  a  wide  range  of  transmission  paths  in  an  interoperable  manner  with  naval,  joint,  coalition  and  civil  / 
law  enforcement  agencies.  Platforms  and  vehicles  communicate  freely  and  autonomously  with  other  elements  of 
the  architecture  thus  the  existence  and  functions  of  the  underlying  network  are  transparent  to  the  warfighter. 


Attribute 

Notional  Measures 

Robustness 

Number  of  differing  conditions/environments  over  which  network  is  capable  of 
operating  at  a  given  level  of  effectiveness  (baseline  level  determined  by  SME, 
simulation,  analysis,  empirical  analysis,  etc.) 

Effectiveness  of  network  across  varying  levels  of  attack/degradation  (baseline  level 
determined  by  SME,  simulation,  analysis,  empirical  analysis,  etc.) 

Number  of  tasks/missions  which  the  network  is  capable  of  operating  at  a  given  level  of 
effectiveness  (baseline  level  determined  by  SME,  simulation,  analysis,  empirical 
analysis,  etc.) 

Responsiveness 

The  timeliness  of  the  response  to  an  environmental  change  (baseline  level  determined 
by  SME,  simulation,  analysis,  empirical  analysis,  etc.) 

Flexibility 

Number  of  options  for  responding  to  an  environmental  change 

Compatibility  of  different  responses  (0  =  not  compatible,  1  =  fully  compatible; 
determined  by  SME,  simulation,  analysis,  empirical  analysis,  etc.) 

Innovativeness 

Number  of  novel  responses  developed  and  implemented  (baseline  determined  by  SME, 
simulation,  analysis,  empirical  analysis,  etc.) 

Adaptiveness 

Number  and  timeliness  of  changes  to  network  structure  and  processes  (baseline 
determined  by  SME,  simulation,  analysis,  empirical  analysis,  etc.) 

4.  Provide  adaptive  /  automated  decision  aids  (1/2) 

To  support  warfighter  decision  making  by  providing  recommended  courses  of  action  that  are  adaptive  and  based 
upon  knowledge  of  the  operational  context,  commander’s  intent,  rules  of  engagement,  order  of  battle,  etc.  and 
evolution  of  the  battlespace  landscape 

Attribute 

Notional  Measures 

Robustness 

Degree  to  which  decision  aids  support  decision  making  across  a  range  of 
situations  and  degradation  conditions 

Responsiveness 

Degree  to  which  decision  aids  support  decision  making  which  is  relevant 
and  timely 

Innovativeness 

Degree  to  which  decision  aids  support  decision  making  that  reflects  novel 
ways  to  perform  known  tasks 

Adaptability 

Degree  to  which  decision  aids  support  a  decision  making  process  with  the 
flexibility  to  alter  decision  making  in  response  to  the  evolution  of  the 
battlespace  landscape 

Consistency 

Extent  to  which  decision  aids  support  decision  making  are  internally 
consistent  with  prior  understanding  and  decisions 

4.  Provide  adaptive  /  automated  decision  aids  (2/2) 

To  support  warfighter  decision  making  by  providing  recommended  courses  of  action  that  are  adaptive  and  based 
upon  knowledge  of  the  operational  context,  commander’s  intent,  rules  of  engagement,  order  of  battle,  etc.  and 
evolution  of  the  battlespace  landscape 

Attribute 

Notional  Measures 

Currency 

Extent  to  which  decision  aids  support  decision  making  that  minimizes  latency 
(e.g.  Notification  -  Time  of  detection  =  Cueing  Time,  Time  of  detection  - 
receipt  of  refined  positional  estimate  =  Update  rate,  Time  of  cueing  data  - 
time  of  weapon  firing  =  weapons  release  time,  Firing  report  received  by  group 
commander  -  weapons  firing  time  =  Firing  report  time) 

Precision 

Error  and  confidence  level  for  time  and  position  information  compared  to  a 
standard  reference 

Fitness  for  Use 

Relative  quality  in  reference  to  criteria  that  are  determined  by  the  situation 

Appropriate¬ 

ness 

Extent  to  which  decision  aids  support  decisions  that  are  consistent  with 
existing  understanding,  command  intent  and  values 

Completeness 

Extent  to  which  decision  aids  support  relevant  decisions  that  encompass  the 
necessary: 

•Depth:  range  of  actions  and  contingencies  included 
•Breadth:  range  of  force  elements  included 

•Time:  range  of  time  horizons  included  I 

5.  Provide  human-centric  integration  (1/2) 

Enhance  the  ability  of  warriors  to  multi-task  through  all  phases  of  warfare  while  taking  advantage  of  improved 
Human-Computer  Interfaces  which  leverage  the  best  of  humans  &  computers.  This  will  be  possible  through 
improved  automation  &  development  of  multi-sensory  human-computer  interfaces.  i 

Attribute 

Notional  Measures 

Competence 

Distribution  of  members  knowledge,  skills,  abilities  and  attitudes. 

Trust 

Extent  to  which  members  are  willing  to  rely  on  one  another 

Confidence 

Extent  to  which  members  have  expectations  of  the  reliability  of  the  organization 

Size 

Number  of  team  members  involved  adequate  to  support  mission 

Experience 

Degree  to  which  team  members  have  interacted  in  the  past  on  the  same  task 

Diversity 

Degree  to  which  team  members  are  heterogeneous  or  homogeneous  across 
exogenous  variables:  experience,  age,  gender,  etc. 

Autonomy 

Extent  to  which  organization  is  externally  or  self  directed 

5.  Provide  human-centric  integration  (2/2) 

Enhance  the  ability  of  warriors  to  multi-task  through  all  phases  of  warfare  while  taking  advantage  of  improved 
Human-Computer  Interfaces  which  leverage  the  best  of  humans  &  computers.  This  will  be  possible  through 
improved  automation  &  development  of  multi-sensory  human-computer  interfaces. 

Attribute 

Notional  Measures 

Structure 

Numbers  of  layers  of  authority 

Functional  Differentiation  Effectiveness 

Interdependence 

Extent  to  which  members  depend  on  one  another  for  resources 

Cooperation 

Extent  to  which  member(s)  are  willing  and  able  to  work  together 

Efficiency 

Extent  to  which  members  utilize  one  another’s  resources  so  as  to 
minimize  costs  and  maximize  benefits 

Synchronization 

Extent  to  which  organization  is  conflicted,  deconflicted,  or  synergistic 

Engagement 

Extent  to  which  all  members  actively  and  continuously  participate 

Risk  Propensity 

Extent  of  risk  aversion  ' 

6.  Provide  information  weapons: 

To  integrate  the  use  of  military  deception,  psychological  operations,  electronic  warfare,  and  physical 
destruction,  mutually  supported  by  intelligence,  in  order  to  deny  information,  influence,  degrade,  or  destroy 
adversary  information,  information-based  processes,  and  information  systems.  (Metrics  are  under  development.) 

Attribute 

Notional  Measures 

Lethality 

Extent  of  capability  to  precisely  deliver  desired  Non-Kinetic  (NK)  Information 
Operations  (10)  effects. 

Coverage 

Extent  of  capability  to  accomplish  10  effects. 

Persistence 

Extent  of  capability  to  sustain  10  effects. 

Timeliness 

Extent  of  capability  to  deliver  desired  NK  10  effects  at  a  desired  time. 

Survivability 

Extent  of  capability  to  avoid  enemy  threats,  counter  ISR,  and  employ  IO  1 

techniques  to  reduce  targeting  of  adversary  kinetic  systems  allowing  increased 
secure  maneuvering  by  ASMD/Deny  ISR/SEAD/Networks. 

Ontology:  Levels  of  Shared 
Situational  Awareness  Define 


Level 

SA  Focus 

SA  Process 

5 

Trans-formational 

Agent  based  communication; 
establishing  a  process  for 
adapting  processes  to  support 
operational  contingencies; 
establishing  knowledge 
delivery  mechanisms  to 
provide  knowledge  to  strategic 
partners;  process  optimization 

•Evaluation  of  performance  and  effectiveness  on  a 
continuous  basis 

•Identifying  adjustments  and  potential  improvement  to 
the  fusion  process 

•Determining  source  specific  data  requirements  for 
processing 

Recommending  allocation  and  direction  of  resources  in 
support  of  the  mission 

Understanding  mission,  opportunities  and  risks, 
adversary’s  capabilities  and  limitations,  analysis  of 
possible  outcomes,  and  adversary’s  intent 

4 

Predictable 

Concepts  embedded  in  data 
translated  into  a  common 
ontology;  data  mining  for 
patterns  and  relationships; 
presentation  of  knowledge 
based  upon  the  user’s  learning 
profile;  network  of  multiple 
portals  enabling  the  real-time 
aggregation  of  disparate 
knowledge 

•Estimating  capabilities,  i.e.  number  and  location 
•Predicting  enemy  intent  based  on  actions, 
communications  and  enemy  doctrine 
•Identifying  threat  opportunities  -  ID  of  potential 
opportunities  for  enemy  threat 
•Assessment  from  multi-perspectives 
•Analyzing  predictions  of  offensive/defensive  results  of 
hypothesized  engagements 

•Understanding  mission,  opportunities  and  risks, 
adversary’s  capabilities  and  limitations,  analysis  of 
possible  outcomes 

Ontology:  Levels  of  Shared 
Situational  Awareness  Defined 


Level 

SA  Focus 

SA  Process 

3 

Defined 

Data  is  aggregated  in  a 
central  data  base; 
data  from  multiple  operational 
systems  can  be  extracted  on  demand; 
richer  artifacts  of  the  process  are 
stored  and  organized;  data 
presentation  includes  summaries  and 
analysis; 

collaborative  tools  which  capture  the 
timeliness,  breadth  and  depth  of 
subject  matter  experts 

•Estimating  relationships  among  aggregated  objects 
including  events/activities 

•Interpreting  within  context  weather,  terrain  and  other 
environmental  considerations 

•Assessment  from  a  multi-perspective  (i.e.  Blue,  Red  & 

White  viewpoints) 

•Understanding  mission,  opportunities  and  risks,  adversary's 
capabilities  and  limitations 

2 

Repeatable 

Data  repository  mechanism  provided 
to  capture  individual  input  and 
retrieve  data;  forum  provided  for 
distributed  collaboration 

Focus  on  individual  objects 

Associating  sensor  outputs  w/specific  known  objects  or 
initiate  new  objects 

Using  sensor  data  to  refine  the  best  estimates  of  current 
positions  for  each  hypothesized  object. 

Understanding  mission,  opportunities  and  risks 

1 

Initial 

Limited  collaboration,  data  fusion  or 
correlation 

•Aligning  data  with  respect  to  time/space 
•Relating  newly  received  observations  to  existing  track 
•Comprehending  basic  classification  of  emitters,  platforms, 
etc. 

•Understanding  mission 

Case  Study:  FORCEnet  MC02  Data  Analysis 

Data  was  collected  in  MC02  to  determine  the  impact  of  values 
on  organizational  effectiveness. 

Values  for  the  study  included:  Centralization,  Formalization, 
Interdependence,  Reciprocity  and  Trust.  Organizational 
effectiveness  was  based  on  a  Situational  Awareness  Maturity 
Assessment. 

Final  results  are  pending  detailed  and  statistical  analysis. 

Preliminary  Conclusions: 

Traditional  cultural  values  are  inconsistent  with  information 
sharing  and  are  barriers  to  Network  Centric  Warfare. 

Based  upon  the  research  this  has  implications  for  the 
management  of  distributed  teams. 


Implications  for  Management 

>  Collaboration  and  cooperation  become  key  enablers  for 
organizations  to  share  and  utilize  the  knowledge  and 
expertise  of  their  human  capital. 

>  Motivation  and  reward  systems  must  be  revamped  to  provide 
incentives  for  innovation;  sharing  and  utilizing  knowledge. 

>  Organizations  evolve  into  less  hierarchical  structures  as 
networked  organizations  develop  that  are  flexible  and  able  to 
adapt  quickly. 

>  Group  and  team  working  enhance  communication  and 
knowledge  sharing,  and  utilize  the  best  skills  and  experience 
on  every  task 

>  Values  of  the  organization  change  to  reflect  the  reliance  on 
people,  knowledge,  and  information 


Summary 

C4ISR  Value  assessment  depends  upon  decomposition  and 
contextual  definition  of  the  tradespace 

Determining  the  value  of  capabilities  requires  an  integrated 
analytical  framework  with  supporting  measures/metrics 

Current  FORCEnet  capabilities  definition  provide  a  solid 
taxonomic  foundation  for  establishing  that  framework 

Contextual  analysis  and  trade  decisions  require  situation 
awareness  and  methodologies  to  ensure  alignment  of 
perspectives 

A  Situation  Awareness  Capabilities  Maturity  Model 
approach  provides  an  ontological  resource  for  assessment 
using  the  capabilities  taxonomy. 
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FORCEnet  Overview 


“FORCEnet  is  the  operational 
construct  and  architectural 
framework  for  Naval  Warfare  in  the 
Information  Age  which  integrates 
Warriors,  sensors,  networks, 
command  and  control,  platforms  and 
weapons  into  a  networked, 
distributed  combat  force,  scalable 
across  the  spectrum  of  conflict  from 
seabed  to  space  and  sea  to  land.  ” 


FORCEnet  is  not: 
•A  Program 
•A  System  or  thing 
•An  end-state 
•“Just  a  net” 


FORCEnet  is: 

•Mechanism  for  Naval  transformation 
•Enterprise  alignment  and  integration  effort 
•Path  to  make  NCW  a  reality 
•Mechanism  for  Warrior  Enablement 
•Capabilities-based  approach  to  requirements 
and  investment,  looking  across  stovepipes 


FORCEnet  Capability  Growth 
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♦  =  Desired  “end  state”  for  each  capability 

(value)  =  Weight  in  warfighting  outcomes  (N6/N7  BCAPP  scenarios) 
-  =  notional  status  of  capability 


